Amendments to the Claims: 

This listing of claims will replace all prior versions, and listings, of claims in the 
application: 

Listing of Claims: 

1 (previously presented). A method for structuring video by 
probabilistic merging of video segments, said method comprising the steps of: 

a) obtaining a plurality of frames of unstructured video; 

b) generating video segments from the unstructured video by 
detecting shot boundaries based on color dissimilarity between consecutive 
frames; 

c) extracting a feature set by processing pairs of said segments, 
said extracting generating an inter-segment color dissimilarity feature and an 
inter-segment temporal relationship feature of each said pair of segments, said 
inter-segment temporal relationship feature including metrics of temporal 
separation between the segments of the respective said pair and accumulated 
duration of the segments of the respective said pair; and 

d) merging video segments with a merging criterion that applies a 
probabilistic analysis to the features of the feature set, thereby generating a 
merging sequence representing the video structure. 

2 (cancelled). 

3 (previously presented). The method as claimed in claim 23 
wherein the difference signal is based on a mean dissimilarity determined over a 
plurality of frames centered on one of the consecutive frames. 

4 (previously presented). The method as claimed in claim 23 
ftirther including the step of morphologically transforming the threshold 
difference signal with a pair of structuring elements that eliminate the 
presence of multiple adjacent shot boundaries. 
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5 (original). The method as claimed in claim 1 wherein the 
processing of pairs of segments for visual dissimilarity in step c) comprises 
the steps of computing a mean color histogram for each segment and 
computing a visual dissimilarity feature metric from the difference between 
mean color histograms for pairs of segments. 

6 (previously presented). The method as claimed in claim 7 
wherein the processing of pairs of segments for their temporal relationship in 
step c) comprises the processing of pairs of segments for a temporal 
separation between pairs of segments and for an accumulated temporal 
duration of pairs of segments. 

7 (previously presented). A method for structuring video by 
probabilistic merging of video segments, said method comprising the steps 
of; 

a) obtaining a plurality of frames of unstructured video; 

b) generating video segments from the unstructured video by 
detecting shot boundaries based on color dissimilarity between consecutive 
frames; 

c) extracting a feature set by processing pairs of said segments, 
said extracting generating an inter-segment color dissimilarity feature and an 
inter-segment temporal relationship feature of each said pair of segments; and 

d) merging video segments with a merging criterion that applies 
probabilistic analysis to the features of the feature set, thereby generating a 
merging sequence representing the video structure; 

wherein step d) comprises the steps of: 

generating parametric mixture models to represent 
class-conditional densities of inter-segment features of the feature set, 
said parametric mixture models being statistical models; and 

applying the merging criterion to the parametric 
mixture models. 

8 (original). The method as claimed in claim 7 wherein step 
d) is performed in a hierarchical queue and comprises the steps of: 



initializing the queue by introducing each feature into the 
queue with a priority equal to the probability of merging each corresponding 
pair of segments; 

depleting the queue by merging the segments if the merging 
criterion is met; and 

updating the model of the merged segment and then updating 
the queue based upon the updated model. 

9 (original). The method as claimed in claim 1 wherein 
representing the merging sequence is represented in a hierarchical tree stmcture. 

10 (previously presented). A computer storage medium having 
instructions stored therein for causing a computer to perform the acts of: 

generating video segments from unstructured video by detecting 
shot boundaries based on color dissimilarity between consecutive frames; 

extracting a feature set by processing pairs of segments, said 
extracting generating an inter-segment color dissimilarity feature and an inter- 
segment temporal relationship feature of each said pair of segments; and 

merging video segments with a merging criterion that applies a 
probabilistic analysis to the features of the feature set, thereby generating a 
merging sequence representing the video structure; 

wherein said merging further comprises the steps of: 

generating statistical models of the feature set; and 
applying the merging criterion to the statistical models. 



1 1 (previously presented). A method for structuring video by 
probabilistic merging of video segments, said method comprising the steps 
of: 

a) obtaining a plurality of frames of xmstructured video; 

b) generating video segments from the unstructured video by 
detecting shot boundaries based on color dissimilarity between consecutive video 
frames; 
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c) extracting a feature set by processing pairs of segments, said 
extracting generating an inter-segment color dissimilarity feature and an inter- 
segment temporal relationship feature of each said pair of segments; 

d) generating a parametric mixture model of the inter-segment 
features comprising the feature set, said parametric mixture model being a 
statistical model; and 

e) merging video segments with a merging criterion that applies 
probabilistic Bayesian analysis to the parametric mixture model, thereby 
generating a merging sequence representing the video structure. 

12 (original). The method as claimed in claim 1 1 wherein the 
processing of pairs of segments for visual dissimilarity in step c) comprises 
the steps of computing a mean color histogram for each segment and 
computing a visual dissimilarity feature metric from the difference between 
mean color histograms for pairs of segments. 

13 (previously presented). The method as claimed in claim 1 1 
wherein the processing of pairs of segments for their temporal relationship in 
step c) comprises the processing of pairs of segments for a temporal 
separation between pairs of segments and for an accumulated temporal 
duration of pairs of segments. 

14 (original). The method as claimed in claim 1 1 wherein the 
parametric mixture model generated in step d) represents class-conditional 
densities of the inter-segment features comprising the feature set. 

15 (original). The method as claimed in claim 1 1 wherein step 
e) is performed in a hierarchical queue and comprises the steps of: 

initializing the queue by introducing each feature into the 
queue with a priority equal to the probability of merging each corresponding 
pair of segments; 

depleting the queue by merging the segments if the merging 
criterion is met; and 



updating the model of the merged segment and then updating 
the queue based upon the updated model. 

16 (original). The method as claimed in claim 1 1 wherein the 
merging sequence is represented in a hierarchical tree structure that includes a 
frame extracted from each segment and displayed at each node of the tree. 

1 7 (previously presented). A computer storage medium having 
instructions stored therein for causing a computer to perform acts for structuring 
video by probabilistic merging of video segments, the acts including: 

obtaining a plurality of frames of unstructured video; 

generating video segments from the unstructured video by 
detecting shot boundaries based on color dissimilarity between consecutive video 
frames; 

extracting a feature set by processing pairs of segments, said 
extracting generating an inter-segment color dissimilarity feature and an inter- 
segment temporal relationship feature of each said pair of segments; 

generating a parametric mixture model of the inter-segment 
features comprising the feature set, said parametric mixture model being a 
statistical model; and 

merging video segments with a merging criterion that applies a 
probabilistic Bayesian analysis to the parametric mixture model, thereby 
generating a merging sequence representing the video structure. 

1 8 (previously presented). A method for structuring video by 
probabilistic merging of video segments, said method comprising the steps of: 

a) obtaining a plurality of frames of unstractured video; 

b) generating video segments from the unstructured video by 
detecting shot boundaries based on color dissimilarity between consecutive video 
frames; 

c) extracting a feature set by processing pairs of segments, said 
extracting generating an inter-segment color dissimilarity feature and an inter- 
segment temporal relationship feature of each said pair of segments; 



d) merging adjacent video segments with a merging criterion that 
appUes a probabilistic Bayesian analysis to parametric mixture models derived 
from the feature set, said parametric mixture models being statistical models, 
thereby generating a merging sequence; and 

e) representing the merging sequence in a hierarchical tree 

structure. 

19 (original). The method as claimed in claim 18 wherein 
representing the merging sequence in a hierarchical tree structure includes 
displaying a frame extracted from each segment. 

20 (previously presented). A computer storage medium having 
instructions stored therein for causing a computer to perform probabilistic 
merging of video segments, said instructions performing the acts of: 

a) obtaining a plurality of frames of unstructured video; 

b) generating video segments from the unstructured video by 
detecting shot boundaries based on color dissimilarity between consecutive video 
frames; 

c) extracting a feature set by processing pairs of segments, said 
extracting generating an inter-segment color dissimilarity feature and an inter- 
segment temporal relationship feature of each said pair of segments; 

d) merging adjacent video segments with a merging criterion that 
applies a probabilistic Bayesian analysis to parametric mixture models derived 
from the feature set, said parametric mixture models being a statistical models, 
thereby generating a merging sequence; and 

e) representing the merging sequence in a hierarchical tree 

structure. 

21 (previously presented). A method for structuring video by 
probabilistic merging of video segments, said method comprising: 

generating video segments from an unstructured plurality of video 
frames by detecting shot boundaries based on color dissimilarity between 
consecutive frames; 
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extracting a feature set by processing pairs of segments, said 
extracting generating an inter-segment color dissimilarity feature and an inter- 
segment temporal relationship feature of each said pair of segments, said inter- 
segment temporal relationship feature including metrics of temporal separation 
between the segments of the respective said pair and accumulated duration of the 
segments of the respective said pair; 

merging the video segments with a merging criterion that applies a 
probabilistic analysis to the feature set, thereby generating a merging sequence 
representing the video structure, the merging being independent of any empirical 
parameter determination; and 

generating a hierarchy with the merged video segments, the 
hierarchy having a merging sequence represented by a binary partition tree. 

22 (previously presented). The method as claimed in claim 21 
wherein the merging the video segments includes maximizing the a posteriori 
probability mass function of a binary random variable that represents inter- 
segment features of the video segments. 

23 (previously presented). The method as claimed in claim 1 
wherein step b) comprises the steps of: 

generating color histograms from the consecutive frames; 

generating a difference signal from the color histograms that 
represents the color dissimilarity between consecutive frames; and 

thresholding the difference signal based on a mean dissimilarity 
determined over a plurality of frames, thereby producing a signal that indicates an 
existence of a shot boundary. 

24 (previously presented). The method as claimed in claim 1 
wherein said extracting of said inter-segment temporal relationship feature of each 
said pair of segments including determining a number of frames separating the 
respective said pair of segments and determining an accumulated number of 
frames in said segments of the respective said pair of segments. 
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25 (previously presented). The method as claimed in claim 1 
wherein step d) comprises the steps of: 

generating parametric mixture models to represent class- 
conditional densities of inter-segment features of the feature set, said 
parametric mixture models being statistical models; and 

applying the merging criterion to the parametric mixture 

models. 

26 (previously presented). The method as claimed in claim 25 
wherein step d) is performed in a hierarchical queue and comprises the steps 
of: 

initializing the queue by introducing each feature into the 
queue with a priority equal to the probability of merging each corresponding 
pair of segments; 

depleting the queue by merging the segments if the merging 
criterion is met; and 

updating the model of the merged segment and then updating 
the queue based upon the updated model. 

27 (previously presented). A method for structuring video by 
probabilistic merging of video segments, said method comprising the steps of: 

generating video segments from a plurality of frames of 
unstructured video by detecting shot boundaries based on color dissimilarity 
between consecutive frames; 

computing an inter-segment color dissimilarity feature and an 
inter-segment temporal relationship feature of each said pair of segments, said 
inter-segment temporal relationship feature including metrics of temporal 
separation between the segments of the respective said pair and accumulated 
duration of the segments of the respective said pair; and 

d) merging video segments with a merging criterion that applies 
probabilistic analysis to said features, thereby generating a merging sequence 
representing the video structure. 



28 (previously presented). The method of claim 27 wherein said 
computing of said inter-segment temporal relationship feature of each said pair of 
segments further comprises determining a number of frames separating the 
respective said pair of segments and determining an accumulated number of 
frames in said segments of the respective said pair of segments. 

29 (previously presented). A method for structuring video by 
probabilistic merging of video segments, said method comprising the steps of: 

obtaining a plurality of frames of unstructured video; 

generating video segments from the unstructured video by 
detecting shot boundaries based on color dissimilarity between consecutive 
frames; 

extracting an inter-segment color dissimilarity feature and an inter- 
segment temporal relationship feature of each said pair of segments, said 
extracting of said inter-segment temporal relationship feature of each said pair of 
segments including determining a number of frames separating the respective said 
pair of segments and determining an accumulated number of frames in said 
segments of the respective said pair of segments; and 

merging video segments with a merging criterion that applies a 
probabilistic analysis to the features of the feature set, thereby generating a 
merging sequence representing the video structure. 
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